fix(dagster-dbt): improve error messages for missing dbt adapter and invalid target_path#33924
Conversation
…invalid target_path - Wrap `dbt.adapters` import in `compat.py` with an actionable ModuleNotFoundError that suggests installing the correct adapter package - Add the same ModuleNotFoundError guard in `_initialize_dbt_core_adapter` for the `dbt.adapters.factory` import - Split the bare `except:` in `cli()` adapter init into a specific ModuleNotFoundError branch with an installation hint, and an Exception fallback for other errors - Add an explicit isinstance check for `target_path` in `cli()` to surface a clear ValueError instead of an AttributeError when a str is passed
Greptile SummaryThis PR improves developer-facing error messages in
Confidence Score: 5/5Safe to merge — both changed files introduce purely additive error-handling logic that falls back to the original behaviour on any unexpected path. All new code paths either raise an exception earlier with a better message (target_path guard) or convert a bare except into a typed handler that still swallows the error and continues (adapter init). No new code alters successful-path behaviour, mutates shared state, or risks data loss. The heuristic "dbt.adapters" in str(e) correctly matches adapter-plugin ModuleNotFoundErrors in dbt 1.x, and the entire adapter-init block is already skipped for dbt 2.x (fusion engine), so the condition cannot fire on an unexpected code path. compat.py warrants a second look: the install hint names platform-specific adapter packages (dbt-postgres, dbt-bigquery, dbt-snowflake) for an import that lives in dbt-adapters, a core dependency of dbt-core itself, so the guidance may point users in the wrong direction when they already have those adapters installed.
|
| Filename | Overview |
|---|---|
| python_modules/libraries/dagster-dbt/dagster_dbt/compat.py | Wraps the dbt.adapters.base.impl import in a try/except to surface a user-friendly error; the error message directs users to install platform-specific adapters even though this module lives in dbt-adapters (a dbt-core dependency), which could misdirect users who already have those adapters installed. |
| python_modules/libraries/dagster-dbt/dagster_dbt/core/resource.py | Adds an isinstance guard on target_path to raise a clear ValueError for string inputs, and replaces the bare except clause around _initialize_dbt_core_adapter with a split ModuleNotFoundError / Exception handler that shows an installation hint when "dbt.adapters" appears in the error message. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[dagster_dbt import] --> B{dbt-core installed?}
B -- No --> C[DBT_PYTHON_VERSION = None\nno imports attempted]
B -- Yes --> D[try: import dbt.adapters.base.impl]
D -- success --> E[import NodeStatus / NodeType etc.]
D -- ModuleNotFoundError --> F[raise ModuleNotFoundError\nwith install hint\ne.g. dbt-postgres / dbt-bigquery]
G[DbtCliResource.cli called] --> H{target_path is a Path?}
H -- No / str passed --> I[raise ValueError\nwith Path example]
H -- Yes / None auto-generated --> J{dbt CLI version < 2?}
J -- No / fusion engine --> K[skip adapter init\nreturn DbtCliInvocation]
J -- Yes --> L[_initialize_dbt_core_adapter]
L -- success --> K
L -- ModuleNotFoundError\ncontains dbt.adapters --> M[logger.warning\ninstall hint]
L -- ModuleNotFoundError\nother --> N[logger.warning\ngeneric + exc_info]
L -- Exception --> N
M --> K
N --> K
Reviews (2): Last reviewed commit: "fix(dagster-dbt): surface actionable err..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| except ModuleNotFoundError as e: | ||
| if "dbt.adapters" in str(e) or "dbt-" in str(e): | ||
| logger.warning( | ||
| "A dbt adapter package could not be loaded. Please install the" | ||
| " appropriate dbt adapter for your data platform (for example:" | ||
| " dbt-postgres, dbt-bigquery, or dbt-snowflake). Some Dagster" | ||
| " features that require a live connection to your data platform" | ||
| " will be unavailable.\n\nOriginal error: %s", | ||
| e, | ||
| ) | ||
| else: | ||
| logger.warning( | ||
| "An error was encountered when creating a handle to the dbt adapter in Dagster.", | ||
| exc_info=True, | ||
| ) |
There was a problem hiding this comment.
_initialize_dbt_core_adapter now raises a ModuleNotFoundError whose message already includes "Original error:\n{e}". When cli() then catches that exception and logs it as "...\n\nOriginal error: %s", e, the final warning prints the installation hint twice and "Original error" twice in nested form. A user would see something like:
A dbt adapter package could not be loaded … Original error: A dbt adapter package could not be found … Original error: No module named 'dbt.adapters.factory'
Additionally, the condition "dbt.adapters" in str(e) or "dbt-" in str(e) always evaluates to True for exceptions raised by _initialize_dbt_core_adapter, because the new custom message itself contains "dbt-postgres", "dbt-bigquery", and "dbt-snowflake". The else branch (lines 744–748) is effectively dead code on that specific error path, leaving the routing logic fragile—if the message in _initialize_dbt_core_adapter is ever changed, the routing silently breaks.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
| raw_target_path = target_path or self._get_unique_target_path(context=context) | ||
| if not isinstance(raw_target_path, Path): | ||
| raise ValueError( | ||
| f"The 'target_path' argument must be a pathlib.Path, not {type(raw_target_path).__name__!r}." | ||
| " Pass a Path object, for example: target_path=Path('target')." | ||
| ) | ||
| target_path = raw_target_path |
There was a problem hiding this comment.
raw_target_path variable is assigned and then immediately reassigned to target_path without any transformation. The isinstance guard can operate on target_path directly.
| raw_target_path = target_path or self._get_unique_target_path(context=context) | |
| if not isinstance(raw_target_path, Path): | |
| raise ValueError( | |
| f"The 'target_path' argument must be a pathlib.Path, not {type(raw_target_path).__name__!r}." | |
| " Pass a Path object, for example: target_path=Path('target')." | |
| ) | |
| target_path = raw_target_path | |
| target_path = target_path or self._get_unique_target_path(context=context) | |
| if not isinstance(target_path, Path): | |
| raise ValueError( | |
| f"The 'target_path' argument must be a pathlib.Path, not {type(target_path).__name__!r}." | |
| " Pass a Path object, for example: target_path=Path('target')." | |
| ) |
| try: | ||
| from dbt.adapters.base.impl import ( | ||
| BaseAdapter as BaseAdapter, | ||
| BaseColumn as BaseColumn, | ||
| BaseRelation as BaseRelation, | ||
| ) | ||
| except ModuleNotFoundError as e: | ||
| raise ModuleNotFoundError( | ||
| "A dbt adapter package could not be found.\n\n" | ||
| "Please install the appropriate dbt adapter for your data platform " | ||
| "(for example: dbt-postgres, dbt-bigquery, or dbt-snowflake).\n\n" | ||
| f"Original error:\n{e}" | ||
| ) from e |
There was a problem hiding this comment.
dbt.adapters.base.impl lives in the dbt-adapters package, which is a core dependency of dbt-core, not a specific adapter package. If this import fails with ModuleNotFoundError, it means dbt-adapters itself is absent (e.g., a broken or partial install), not that a platform-specific adapter like dbt-postgres is missing. Installing dbt-postgres would indirectly fix the issue (it pulls in dbt-adapters), but a user who already has dbt-postgres installed would be confused by being told to install it again. The message would be more accurate if it mentioned dbt-adapters as the primary fix, with platform adapters as the second suggestion.
…nd invalid target_path - In `compat.py`, wrap the `dbt.adapters.base.impl` module-level import with a `ModuleNotFoundError` handler that tells users which adapter package to install (e.g. dbt-postgres, dbt-bigquery, dbt-snowflake) - In `cli()`, replace the bare `except:` with a split handler: a `ModuleNotFoundError` branch that emits a targeted installation hint when `dbt.adapters` is missing, and an `Exception` fallback that preserves the original generic warning for unrelated failures - Add an explicit `isinstance(target_path, Path)` guard in `cli()` that raises a clear `ValueError` instead of a cryptic `AttributeError: 'str' object has no attribute 'is_absolute'`
1 participant
Summary & Motivation
dagster-dbtcurrently surfaces common setup failures as low-level Pythonexceptions with no guidance on how to fix them. This PR improves the
developer experience by replacing those raw errors with actionable messages.
Changes:
compat.py): wraps thedbt.adapters.base.implimport in atry/except ModuleNotFoundErrorthat tells users exactly which package to install (e.g.
dbt-postgres,dbt-bigquery,dbt-snowflake)._initialize_dbt_core_adapter):applies the same guard to the
dbt.adapters.factoryimport so thehelpful message is shown regardless of which code path triggers the error.
cli()): replaces the bareexcept:with asplit handler — a
ModuleNotFoundErrorbranch that emits a targetedinstallation hint, and an
Exceptionfallback that preserves theoriginal generic warning for unrelated failures.
target_pathtype (cli()): adds an explicitisinstancecheck that raises a clear
ValueErrorwhen astris passed instead ofa
Path, preventing the confusingAttributeError: 'str' object has no attribute 'is_absolute'.Test Plan
pip uninstall dbt-postgres)and confirm the new
ModuleNotFoundErrormessage is shown at import timeand during
DbtCliResource.cli().target_path="target"(a string) toDbtCliResource.cli()andconfirm a
ValueErrorwith the descriptive message is raised instead ofan
AttributeError.Changelog
Improved error messages in
dagster-dbtfor missing dbt adapter packagesand invalid
target_patharguments. Users now receive actionable guidance(e.g. which adapter package to install) instead of raw Python exceptions.